Facial expression measurement has become extremely valuable in numerous fields such as performance driven facial animation in media production, behaviour analysis, cognitive studies, product design analytics, and measuring emotional responses to advertising.
Methods heretofore developed for measuring facial expressions are typically limited to professional applications insofar as they require constrained capture environments, specialized hardware, trained operators, physical facial markers and long setup times. These requirements make such technologies impractical for some professional and most consumer applications.
If, however, requirements for specialised equipment and environments were to be overcome, measuring facial expressions from video would become well-suited to consumer-based applications. Video cameras are commonly integrated into many consumer electronics, such as mobile phones, laptops, tablet computers, game consoles and televisions and therefore are already available to a large proportion of the consumer population. Measurement of facial expression from video is much less intrusive than other methods such as optical facial motion capture, for example, in US Patent Application 2011/0110561 A1 to Havaldar, entitled “Facial Motion Capture Using Marker Patterns That Accommodate Facial Surface,” which require subjects to have several dozen physical markers applied to the face.
Methods for tracking facial features from video exist in the prior-art. For example, Active Appearance Models (AAMs) as taught, for example, in Cootes, et al., “Active appearance models”, Proc. European Conf. on Computer Vision, vol. 2, pp. 484-98, (Springer, 1998) (hereinafter, “Cootes, et al., (1998), incorporated herein by reference), are a form of tracker which captures the variation in position and appearance of facial features in a training set of example images, and apply these to the tracking of facial features of subjects not included in the training set. If the identity of the target subject for analysis is unknown, these methods generally involve training a generic tracker that will attempt to work on the entire population. This invariably results in the system failing or underperforming on many specific individuals due to the models inability to generalise to the entire population. It is possible to retrain the tracker to include these specific individuals but this is impractical for many applications due the length of time and complexity of the training process.